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ABSTRACT 

The BEES (Bio-inspired Engineering for Exploration Systems) 
for Mars project at NASA Ames Research Center has the goal 
of developing bio-inspired flight control strategies to enable 
aerial explorers for Mars scientific investigations. This paper 
presents a summary of our ongoing research into biologically 
inspired system designs for control of unmanned autonomous 
aerial vehicle communities for Mars exploration. First, we 
present cooperative design considerations for robotic explorers 
based on the holarchical nature of biological systems and 
communities. Second, an outline of an architecture for 
cognitive decision making and control of individual robotic 
explorers is presented, modeled after the emotional nervous 
system of cognitive biological systems. 

Keywords: Holarchy, Biologically Inspired, Emotional UAV 
Flight Control 

1. INTRODUCTION 

The BEES (Bio-inspired Engineering for Exploration Systems) 
for Mars project at NASA Ames Research Center has the goal 
of developing bio-inspired flight control strategies to enable 
aerial explorers for Mars scientific investigations [1]. The 
research presented here will focus on unmanned autonomous 
aerial vehicles (UAVs) for Mars exploration. The significant 
advantage afforded by UAVs over ground-based robotic 
explorers has generated recent interest and research into 
understanding and overcoming the engineering challenges of 
fielding flight vehicles in the Martian atmosphere [2]-[7]. 
These advantages include a higher degree of mobility, 
instrument access to areas that cannot be traversed on the 
ground, and coverage of a larger area of the Martian surface [8]- 
[10]. Section 2 presents ongoing BEES for Mars 
implementation. 

Biologists address the natural world in levels of functional 
groupings, each emergent from its component levels. In this 
paper, we present our work on bio-inspired technology for 
robotic Martian explorers. In functional decomposition of bio- 
inspired concepts, each layer of structure and function mimics a 
biological counterpart. While typical applications of biological 
inspiration ideas tend to isolate a single aspect of the natural 
world for emulation, such as a neural network model of brain 
function [11][12] or swarm concepts based on insect colonies 
[13], the BEES for Mars project integrates multiple levels of 
bio-inspiration. In section 3, elements and classifications of the 
biological holarchy are defined, and their applicability to design 
of robotic communities is discussed. The control and decision 
making of aerial explorers is designed after cognitive models of 
emotion in biological organisms. While the role of emotions in 
the cognitive process is still a hotly debated area of research in 
scientific fields such as neuroscience and psychology, research 


suggests that emotions are the fundamental basis for rational 
decision making by organisms, and provides adaptability in 
making decisions when faced with uncertainty [14]-[17]. 

Through the modeling of emotions, this endeavor attempts to 
capture the essence of rational and adaptive decision making in 
organisms, with the aim of quantifiably outperforming 
unemotional counterparts in goal-oriented tasks when faced 
with uncertainty. Adaptive ability is critical in a remote 
environment where real-time human control of flight vehicles is 
not possible. In such scenarios, the inability for the system to 
adapt quickly to unpredicted adverse situations and other 
uncertainty will result in the unrecoverable loss of the aircraft 
resource. Section 4 will present a defense for utilizing an 
emotional model for flight controls, outline previous work in 
this field, and present an architecture that is currently under 
development that utilizes an emotional nervous system to 
control all aspects of UAV behavior, from decision making to 
navigation and low level automatic controls. Section 5 will 
outline the development strategy and road map for the 
implementation of these research ideas. 

2. FIRST STEP: MISSION CONCEPTS & BIO-INSPIRED 
BEHAVIORS 

Preliminary work on the BEES for Mars effort focused on the 
definition of bio-inspired mission concepts and flight 
“behaviors” that would successfully effect Mars aerial explorer 
demonstrations with terrestrial surrogate vehicles. 

Mission concepts in [18] derive from situations in the biological 
world that are oriented toward “search and find” requirements: 
a mission using dropped aerial probes based on dissemination 
and survivorship curves and a terrain-influenced search 
trajectory derived from predation strategies. A small 
compendium of bio-inspired behaviors was defined for an aerial 
vehicle. These behaviors were categorized into general groups: 
UAV “primitive” tasks, observations, actions, and planning. 
The use of stochastic search strategies was also emphasized in 
this preliminary work. 

Figure 1 illustrates an early BEES for Mars implementation 
demonstrating conventional search and find behaviors. The 
demonstration used a 2-meter wingspan UAV system that had 
been enhanced to include a ground based adaptive decision 
component. This decision component allowed the aircraft 
command sequence to be altered based on images that were 
recognized from video downlinked from the aircraft. 

The mission entailed searching for a target (in this case a large 
orange tarp on the ground), visually recognizing the target, and 
dropping a small aerial imaging probe at/onto the position of the 
target, before resuming its search. 



Figure 1. Early Search and Find Behavior Demonstrations 


Definition of flight behaviors for Mars aerial explorers is only 
the first step towards developing the autonomous system 
technology required for Mars exploration missions. It is also 
necessary to develop techniques to evaluate situations, make 
decisions, and implement behaviors while following the general 
outlines of the mission plan. These requirements will entail 
investigating ways in which intelligent systems evaluate, 
interact and self organize, both individually and with other 
aerial explorers and ground components. 

Again, we turn to nature for concepts that may inspire solutions. 
The use of holarchical design shows promise as an architecture 
in which intelligent systems can interact as a community. Our 
research also shows that decision-making and control in 
intelligent systems may be enhanced by the introduction of 
emotional models. An emotional holon, a rational intelligent 
system capable of interacting within a community of other 
aerial explorers is the goal of our research. 

3. HOLARCHICAL DESIGN CONSIDERATIONS FOR 
ROBOTIC COMMUNITIES 

The words holon and holarchy were coined by Arthur Koestler 
in the 1970’s [19]. A holon expresses a dualistic nature as both 
whole and part; each level of a holarchy functions as an 
independent unit and acts as component of a larger, more 
encompassing whole. In contrast, a hierarchy refers to a 
pyramidal structure of power relationships. Elements of 
hardware or software development are familiarly defined in 
terms of functional hierarchies, where components and sub- 
components contribute to overall system function. In a 
hierarchical architecture, components often lack meaningful 
function outside the context of the larger system function. 
Alternatively, a holarchical architecture approach strives to 
present a balance between near-equal, near-independent but 
interacting, automated agents working across several levels of 
operation and cooperation/interdependence of individual and 
multiple robotic systems. 

The natural world comprises a holarchy, which spans the 
sciences of sub-atomic physics, through chemistry and biology 
to astronomy and cosmology. A molecule is a whole unit, 
whose function emerges from the types, states, and relationships 
among its component atoms, each whole unto themselves. The 
organelles inside a living cell have their own existence as 
structures built of macro-molecules but also contribute to the 
overall life and function of the cell. In the science of biology. 


the holarchy continues through tissues, organs, and organ 
systems inside the individual organism; proceeding outward 
from the organism, the biological holarchy includes 
populations, communities, biomes, culminating in the 
biosphere. 

Biological Community Holarchies 

The designs of autonomous planetary explorers are suited to 
their environment and their function. While biological species 
adapt to their environments through evolution, and robotic 
explorers are created through a structured design process, the 
individual robot is analogous to the organism in the holarchical 
systems approach. 

Nature offers infinite variability and categorical statements can 
only be simplifications or generalizations. A population is a 
group of organisms of the same species living in the same area; 
members of a population interact for reproduction, use of 
resources, protection, social effects, and other purposes. 
Depending on the species, the individuals may compete with 
one another or cooperate but all occupy the generally same 
niche in the ecosystem. In a bio-inspired technological 
holarchy, the population comprises multiple units of the same 
design, serving the same function. 

In BEES for Mars, the number of autonomous aerial vehicles is 
a limited resource and so hardware implementation is 
constrained to one or two vehicles. With continuous vehicle-to- 
ground communication, it is possible to create “software-in-the- 
loop” simulations, where the onboard processor may be fed 
information allowing it to respond to virtual stimuli. Since all 
control algorithms are programmed on desktop processors 
before conversion to flight code, virtual vehicles will behave 
according to the same rules as hardware vehicles. Most of the 
members of a large population will be simulated vehicles. 

In nature, a community is made up of populations of many 
species living in the same area, interacting and using local 
resources. An organizing principle for the study of biological 
communities is the flow of energy through the system. With the 
exception of a few deep-sea thermal vents, the ultimate source 
of all the energy used in the biological world is the sun. There 
are three basic stages in the cycle of energy transformation in 
biological communities, represented by producers, consumers, 
or decomposers. 

Producers are organisms which have the ability to capture the 
incoming energy of the sun and store that energy in the 
chemical bonds of their tissues through photosynthesis. Plants 
are familiar producers in terrestrial ecosystems; trees, for 
example, store great amounts of energy in their wood. Other 
producers are phytoplankton in the ocean, photosynthetic micro- 
organisms on land, and kelp and other algae. Consumers utilize 
the energy stored in other living things for activity and to build 
their own tissues. Primary consumers eat producers, while 
secondary consumers eat other consumers. The “food chain” is 
a familiar metaphor, with lions, zebras, and grasses being a 
favorite example. However, when the varied diets of real 
animals are examined, the relationships among consumers and 
producers form a complex food web. A third and very 
important element in the processing of energy and material in a 
community are the decomposers. Decomposers release the 
energy from dead organisms and organic waste products, 
allowing the materia] to be re-used and taking their own place in 
the food web of consumers. The bacteria that cause meat to rot 
are performing an important function; other examples are dung 


beetles, moths that eat animal fur, and on a larger scale, vultures 
that begin the breakdown of carrion. 

A bio-inspired community of autonomous vehicles, needs a 
variety of “species,” or elements of different designs performing 
a range of functions. In the case of Mars surface exploration, 
example units may be autonomous aerial vehicles, wheeled 
rovers, stationary or onboard processors, portable sensors, or 
orbiting satellites. 

Tn nature, sustamaDie communities result horrrihaividuals^'f' 
the member species performing their particular self-interested 
functions, such as seeking food, shelter, and water, or the 
establishment of territories and mating relationships. Individual 
plants, animals, and fungi are not motivated to maintain 
community balance. Rather, they have co-evolved to perform 
intricately related and complementary functions. In typical 
software control of multiple agents, each unit places at least 
partial value on the state of the overall system in determining its 
own actions. The holarchical approach intends to achieve 
balance in its member functions, without including system state 
in the onboard programming. 

The description of a biological community is based primarily on 
the flow of energy. For a bio-inspired community of 
autonomous vehicles we take an analogy of energy and 
information. The gathering of information may be regarded as 
the motivating purpose for sending autonomous vehicles to 
Mars. The role of information producer can be filled by 
instruments and probes which collect information directly from 
the planetary surface or atmosphere. Information consumers 
take data from producers for processing and may release 
information in a different form to other information consumers. 
Figure 2 illustrates a simple information community. 
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Figure 2. Information Cycle 

Multiple information processing roles may be implemented 
onboard the same hardware platform. The steps in the 
information “food chain” show a similarity to the layers of 
abstraction familiar in robotic software. The decomposer role is 
less distinct with information than with natural material and 
energy. However, as data provide information and the collected 
body of information matures into knowledge, there must be a 
point in the cycle where the decision occurs that a goal is met 
and reinitialization of the collection processes can take place. 
Researchers on Earth may be the ultimate information 
decomposers about the Martian surface. 

The Sub-robot Holarchy 

Vehicle hardware subsystems may be likened to the organ 
systems of living things that provide functions such as structural 
support, locomotion, sensory input, and others. In hardware 






design, as in evolution, form follows function. The hardware in 
the BEES for Mars project uses established aircraft designs with 
no specific attempt to model the natural world. In software, 
however, a new approach to the control function of the 
autonomous vehicle is presented in the emotion-based approach. 

4. EMOTIONAL SYSTEMS 

Research into emotion-based systems is not a ne w field of 
study; previous work demonstrated'tfieTHsibTlitjTof'enJotiOnal' 
control for higher-level cognition and decision-making, 
typically geared towards social emotional behavior and 
mimicking hu man responses. Modeling an emotional nervous 
system for aspects of vehicle behavior and control beyond 
social aspects, however, is an approach whose utility and 
implementation feasibility remain to be demonstrated. 

Unfortunately, the term ’emotion’ is laden with many different 
preconceptions and prejudices. Historically, emotions have 
been characterized as high level cognitive states, as is typically 
seen in theoretical models in psychology. Such states are often 
arrived at through highly complex interactions that are difficult 
to model and which results in erratic human behavior. Many 
have suggested that emotions are an unfortunate heritage from 
animal ancestors which interferes with rational cognition, 
leading to such unequivocal statements as "robots should not be 
equipped with emotions" [20]. In this mindset, the complex 
mechanisms of emotion are considered distinct from and 
competing with cognition, especially reason and rationality. 

Recent research activity and advances in the field of 
neuroscience has begun a radical transition in the way scientists 
are thinking about emotions. Recent findings suggest that 
emotions are a low level conduit for cognition, rather than a 
high level state that is distinct from cognition. Emotions 
provide the mechanism for rational cognitive decision-making 
and adaptive behavior especially significant in uncertain and 
complex environments. This mechanism is fundamental to all 
facets of animal behavior, from low-level instinctual and 
reflexive action to high-level decision making [15-17]. 

This project attempts to capture the mechanism of emotion that 
provides biological organism's adaptive behavior and rational 
cognitive decision making in uncertain and complex 
environments towards achieving goals. Emotion in the context 
of this paper does not refer to high level human emotions such 
as 'happiness' and 'sadness', which are superfluous and probably 
counter-productive to applications in UAV control and robotics 
in general. Rather, this paper presents an outline for a machine- 
based emotional nervous system to control all aspects of the 
autonomous UAV's behavior, from high level decision making 
to low level reflexive behavior, modeled after the emotional 
system of biological organisms. 

Rationale and Goals of this Project 

The motivation for the use of emotions in a goal-oriented 
system rests on three arguments. First, there are benefits to 
deriving inspiration from biological systems. Second, emotions 
play a large role in the adaptive behaviors demonstrated by 
animals and humans. Third, emotions are fundamental to the 
ability of humans in adapting to uncertainties when pursuing a 
goal. In defense of these statements is recognition that nature 
through evolution has had a very long period of time to 
optimize and refine biological designs, and has had time to 
expunge less desirable and detrimental aspects from its systems. 


rational decision making that provides a mechanism for sound 
deductive reasoning as well as prudent reflexive actions, in the 
hopes of providing greater flexibility for machine intelligence. 

Related Work in Emotional Systems 

There have been numerous initiatives to add emotional 
cognition to systems. The entertainment industry has used 
emotional models to provide human like behavior for characters 
in video games [28], The O Z pro ject at CMU [23] utilized an 
emotional model to provide human like "behavior foTinteractive 
fiction and drama. Their approach adds a higher-level emotion 
based cognitive layer (the Em module) above an unemotional 
lower level to close the perceive-think-react loop. This model is 
loosely based on cognitive models of humans described in [25]. 
Ventura [21] contrasts this approach of placing a high level 
emotional layer above a lower level unemotional layer with a 
functional approach that is constructed emotion-based 
throughout. An example of this approach is given in [24], 
where a society of ‘emotion proto-specialist’ agents, each 
associated with a particular emotion, contributes to the 
emergent emotional behavior in a particular way. 


If emotions are considered a hindrance to goal-related tasks, or 
if they are not believed to provide adaptive benefits, then the 
question of implementation for goal oriented systems becomes 
academic. However, there is a growing amount of research that 
supports the use of emotion-based reasoning. Many argue that 
emotions are intimately linked with cognition and perception in 
general, for instance Picard in [14] states that a “critical part of 
our ability to perceive is not logical, but emotional”. Emotions 
are essential to the adaptive qualities of human decision- 
and are ihf . foundation .. nf a - rati onal mind [15 - l fl jSl- 
Ventura et al. [17] [21] point out the trend in current thinking is 
transitioning: from past consideration of emotions as an 
undesirable side effect carried over from our animal ancestors, 
to current beliefs that emotions are a powerful weapon to allow 
quick decision making in complex environments. 

Despite the fact that many disparate models for emotional 
simulation have successfully been implemented, there is general 
recognition that biological components and mechanisms that 
evoke emotional reactions in animals to environmental and 
cognitive stimuli are not well understood [16]. Further, current 
capabilities of computers to process data might still be well 
short of that necessary to simulate such a complete model. The 
best approaches adapt theoretical models of emotion, capturing 
or simulating specific classifications from those models that 
emulate the expected behavior demonstrated by emotional 
organisms. For these categorizations, researchers have drawn 
inspiration from varying fields such as neuroscience, 
physiology, psychology, and philosophy [17]-[27]. 

The development of an emotion-based system requires 
formalization of a consistent model for emotions that is 
practical and machine-implementable. The researcher must 
appreciate the fundamental shortcomings of this endeavor; the 
characterizations will yield a functional description of the 
emotional system, defining and assigning quantifiable values to 
a complex system that is notoriously hard to define and near 
impossible to quantify. This intractability makes broad 
categorical argument on the utility of emotion based systems 
difficult; variant and often conflicting classifications, 
definitions, and implementations yield fundamentally different 
results, and aspects of one approach will not necessary be 
reflected in another. 

The goal of developing a bio-inspired control system for the 
BEES for Mars project is two-fold. First, demonstrate a 
complete emotional model for control of the behavioral and 
decision making processes of an intelligent autonomous aerial 
vehicle with a mission of search and exploration in a remote 
environment. Second, demonstrate the advantage that emotions 
give a system for adaptive cognitive decision making over a 
non-emotional counterpart. 

These goals emphatically exclude the development of an 
explorer system with human-mimicking interfaces, making a 
machine that thinks like a human, or making the explorer 
human-like in appearance or behavior. Rather, successful 
completion of this project will demonstrate a pragmatic 
implementation of a rational emotional nervous system that is 
simple, elegant, mathematically consistent, and that 
demonstrates rational and adaptive decision-making capabilities 
to overcome uncertainty, with eventualities that quantifiably 
outperform an unemotional counterpart towards accomplishing 
the same goals and objectives. This project will illustrate the 
trend to perceive emotions not as an undesirable component of 
biological cognition that hinders and interferes with rational 
thought processes, but rather as a fundamental component of 


In [17], a two-layered system is presented based on aspects 
from several theories on human cognition, including the Canon- 
Bard theory and Papez circuit theory [16]; the system has two 
layers for processing stimuli input: a slower cognitive processor 
which extracts cognitive features of the stimulus to form a 
generalized image model (for instance, the image of a zebra can 
be evaluated as an animal with four legs attached to a body, 
stripped coloring, etc.), and a perceptual processor for more 
basic and immediate instincts that produce a vector of 
desirability (e.g., a lion’s perception of a zebra triggering its 
predatory instincts). The generalized image model is a database 
of information that might be rich, structured, divisible, and 
complex. The vector of desirability contains information that is 
simple, indivisible, and implemented as an ordered list of values 
relating to certain characterizations of the object, such as is it 
positive or negative, desirable or avoidable, edible or inedible, 
etc. The dual representations are used for reasoning purposes, 
where fast reasoning or reflexive actions can use the desirability 
vector, while slower cognition can access the generalized image 
model. A set of complementary mechanisms use data from one 
model to adjust the other. 

McCauley in [22] presents a system based on the psychological 
theory called ‘pandemonium theory’ [26] [27]. In this system, 
each emotion is represented by an agent called a codelet. The 
analogy of an arena is used, with stands, a playing field, and 
sub-arena. A multitude of codelets populate the arena. 
Codelets on the playing field are active, doing whatever they 
were designed to do, while codelets in the stand watch the 
activities of the codelets on the playing field, waiting for 
something to excite them. The level of excitation of a codelet in 
the stand is associated with how loud the codelets yell, which 
also excites other codelets. When excited to a certain level, a 
codelet will activate and move to the playing field to perform its 
action, which will in turn excite other codelets in the stands. 
Codelet actions are linked to other codelets with certain gains 
like links in a neural network. When entering the playing field, 
the sub-arena creates input and output associations between the 
entering codelet and the currently active codelets. This sub- 
arena performs the actual input and output functions of the 
system. The current goal context of the system emerges from 
the active codelets on the playing field. High-level concept 
codelets may remain on the playing field for quite a long time. 



influencing the actions of the whole agent for that time. 
Multiple goal contexts might be competing or cooperating to 
accomplish their tasks. 

An Emotional Architecture for Agent Holons 

The system we propose is fundamentally a functional emotional 
system according to the classification set forth in [21], Our 
approach is based on several different theories, utilizing their 
= s t rengths-for-diff e r e nt-aspects - -oL.vehicle^controh=Mighe& lej4eh 
cognition will be based on psychological theory of 
pandemonium [22] [26] [27], while reflexive actions will utilize 
the dualistic model of the Canon-Bard theory. These 
approaches are combined into a three layered control system: a 
low-level, time critical reflexive control layer, a mid-level 
control layer that commands the reflexive layer (similar to the 
way an autopilot commands the lower level automatic flight 
controls), and a high level cognitive decision layer. 

In implementation, the software is composed of five 
components. A perceptual component provides interaction with 
the outside world. Second, an analytical component provides 
mathematical reasoning. The third component is a cognitive 
component to provide emotion based reasoning. A fourth 
reflexive component provides low-level flight and system 
control. The last component is an output component to allow 
the system to interact with the environment. 



Figure 3. Components of the Architecture. 

The components of the system are shown in Figure 3 above. 
Note that the arrows represent overall data flow to illustrate the 
circular nature of the data and to illustrate the highest-level 
perceive-think-act loop. Other loops exist in this system for the 
different layers of control; for instance, the perceptual-reflexive- 
output components form the tight inner loop for control. These 
loops will be discussed in more detail in the following sections. 

Perceptual Component 

The perceptual component introduces data into the system from 
the external world, or from internal sensors monitoring the 
health and state of the system. Data include measurements from 
sensors, such as accelerometers, thermometers, and GPS, 
communications with other agents or operators through network 
interfaces or other communication hardware, and internal 
software state parameters describing the health of the system. 
Processing may be more or less complex, depending on the 
input and the emotional state of the system. Video images, for 
instance, may be processed by the perceptual components to 
determine signals that are passed to the rest of the system. The 
amount of processing and the availability of data depend on the 


emotional state of the system. Excitation of particular desires 
and anxieties will adjust the relative priorities for processing 
certain components, and can adjust their parameters to magnify 
their effect in the overall system. 

Analytical Component 

The analytical component, as its name suggests, performs most 
of the computational analysis required to drive the emotions of 
rhp system Thp high level emotional state of the system will 
direct which computations need to be performed in order to 
satiate the cognitive component’s need for information, which 
in turn satiates current anxieties and provides input for decision 
making. This component will consume the output from the 
perceptual component to perform its computation, taking part in 
the highest-level data loop. For instance, as an explorer 
ventures farther from an origin point, the anxiety of safely 
returning will increase, prompting the analytical component to 
calculate the amount of fuel required to safely return. This 
input will further excite or relax the emotional anxiety, 
prompting an adjustment of the search vector to reduce the time 
rate of change of the distance from the origin point, or 
triggering an emotional state transition to reflect the fact that all 
search activities should cease and the explorer should return to 
its origin point. 

Cognitive Component 

The cognitive component performs the high level reasoning and 
decision making needed to achieve high level goals. The 
implementation of this component is based on the psychological 
theory of pandemonium, although modified from the earlier 
adaptation by Jackson [27] from the basic theory of Selfridge 
[26]. The basic agents in the system (the demons or codelets) 
represent machine anxieties, which operate in the same arena 
analogy of the basic pandemonium model. Anxieties will be 
excited from the results of analytical calculations performed, 
from perceptual outputs, and from other related anxieties. The 
anxieties that have reached a certain excitation level (and are on 
the playing field) determine the emotional state of the system, 
which will be used to determine the explorer’s state from a 
predefined fuzzy modality. Anxieties, however, do not have 
actions directly associated with them as in the Jackson and 
McCauley adaptation, but rather have an ‘analog’ effect on the 
desires of the explorer depending on excitation level of 
associated anxieties, which in tum affect behavior. For 
instance, the ‘need to return home’ anxiety can cause several 
behavioral modifications to occur, from modification to search 
vectors to a complete state change when sufficiently excited. 

The cognitive component selects and adjusts its course of action 
from its internal emotional state, and its main function will be to 
adjust the reflexive component to effect action. The cognitive 
component will also adjust the analytical and perceptual 
components, directing them to provide computations and 
modify input based on its current anxieties. 

Reflexive Component 

The reflexive component of the system uses the emotional state 
of the system and commands from the cognitive component to 
implement control of the explorer. For UAV, this component 
implements the automatic flight controls of traditional aerial 
systems. The reflexive component forms a tight control loop 
with the perceptual-reflexive-output components, performing 



time critical control. The processing of the reflexive component 
is minimal. 

The reflexive component of the system is based on the dualistic 
neuroscientific theories of Canon-Bard theory and the Papez 
circuit theory. The dualistic model provides fast real-time 
controls based on a vector of desirability, which is analogous to 
the gains in a conventional flight controller. However, we adapt 
the dualistic model for continuous analog input. The input 
comprises of input from GPS, accelerometers, pressure 
inducers, and other sensors needeffTcT sta&ilize~ah(i _ cbntfor 
flight. This data will be fed directly through a feedback loop 
whose gains are determined by the desirability vector, as in a 
typical feedback controlled flight system. This data model will 
also be fed into a generalized model of the data, which will 
accumulate and analyze data with help from the analytical 
component of the system. This dual representation of input (the 
continuous analog input from the sensors) will feed into the 
output for fast reflexive control, and will also feed into specific 
emotional anxieties in the cognitive model that deal with 
controlling the system, forming a higher level control loop 
between the perceptual-reflexive-cognitive components. 

Three Layered Control System 

Emergent from the design of this system is a three-layered flight 
control system: the reflexive layer, the guidance layer, and the 
cognitive layer. The reflexive layer performs the time critical, 
real-time control needed to stabilize and control the UAV. The 
mechanism that enables this functionality is the fast-acting 
desirability vector from the dualistic Canon-Bard model, 
implemented in the tight loop formed from the perceptual- 
reflexive-output components. This loop implements automatic 
stability and control of the aircraft though manipulation of the 
control surfaces. Several desirability vectors are maintained by 
the reflexive system, which will transition from one set to the 
next based on the emotional state of the guidance control layer. 
The desirability vector can be directly modified by the guidance 
layer through a process referred to as marking, and likewise can 
adjust the guidance layer through a process called indexing. 
Constant adjustments to the emotional states through marking 
and indexing provide a mechanism to enable adaptive and 
learning behavior for the control systems in accomplishing its 
desires and goals. 

The guidance layer of the control system encompasses the 
generalized image model, the . dual of the desirability vector 
representation of stimuli from the Canon-Bard model. Since the 
generalized image model is a richer and more complex set of 
observations and descriptions of the stimuli, its complexity 
requires a more elaborate storage scheme and slower access to 
data. This model is sufficient for the less time-critical guidance 
loop, which will perform monitoring and correction of the 
desirability vector. Part of this middle layer will also function 
analogous to a modal autopilot layer, as well as performing task 
oriented data monitoring and evaluation through an arena based 
emotional model. The processed data from the generalized 
model will feed into control-specific anxieties that will in turn 
mark and manipulate the desirability vectors to close the loop. 

The cognitive layer has less timing restrictions than either of the 
two lower layers, and performs the higher level reasoning and 
decision making for the system. The mechanism to support 
reasoning is adapted from the theory of pandemonium. The 
output of this layer provides the high level goals for the system, 
which may include self preservation, long range exploration. 


and target identification. The pandemonium arena will be 
shared between the cognitive and guidance layers, with each 
layer populating the system with codelets for specific purposes. 
Much of the challenge in adapting the arena analogy towards 
flight control will lie in establishing codelet definitions, 
defining how they interact with each other in a network, and 
defining a mechanism for establishing the gains between each 
codelet. 
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Our approach will differ from the majority of previous 
initiatives at introducing emotions into computational systems; 
where as previous approaches have largely either been geared 
towards functions such as mimicking human social behavior in 
computer and machine interfaces or have been subjugated to the 
very highest level reasoning component, this project will strive 
to introduce emotions to an intelligent system for its adaptive 
and reasoning capabilities in a manner which emotions are 
fundamental to operation of all behaviors exhibited, from low 
level automatic control of the agent to high level decision 
making. Further, the system is intended to be scalable from 
individual agents to a communal holarchy of such agents, 
interacting in complex and adaptive manners to achieve an 
overall goal. The eventual goal for interactive communities is 
to demonstrate adaptive emergent behaviors that will help the 
community thrive in the face of unforeseen causality, 
uncertainty, and catastrophe. The initial stage of this endeavor 
will be to demonstrate the utility of emotional systems for single 
agent UAVs, while demonstrating its potential applicability to 
the larger agent holarchy. 

5. APPLICATION TO UNMANNED AERIAL VEHICLES 
Implementation Challenges 

There are several reasons for the interest in developing a 
holarchical approach to the design of autonomous systems and 
applying emotional models to decision-making and control. 
One challenge is how to develop modules and groups of 
modules that will scale well, both within a single aircraft, or 
across multiple aircraft or ground processors. A holarchical 
approach may provide insight into how modules, data, and 
actions are managed in such designs. 

Another challenge is in how to structure the resources in a 
multi-aircraft mission. As alluded to earlier in this paper, much 
of the research in regards to multi-agent behavior has been on 
swarms or swarming behavior. It is strongly believed that what 
is actually needed is a heterogeneous approach, where multiple 
aircraft and ground processors must work together, applying 
different sensors, functions and skill sets to solve a problem. A 
holarchical approach, where aircraft can cooperate with each 
other on multiple levels is the best way to accomplish this goal. 

A third challenge is in how to more effectively inject 
behavioral concepts to the programming of the autonomous 
system. Our application of an emotional paradigm to the 
reasoning and control of the aircraft or multi-aircraft group 
should allow us to examine how it would be developed based on 
behavioral, rather than conventional programming methods. In 
contrast with standard progr ammi ng of if-then else constructs, 
the behavior of the aircraft will be defined by the interaction 
between modules and systems of modules. These interactions 
may be learned via training and experience. 


and information with more than one aerial asset. As a 
steppingstone to multiple UAV autonomy, we will add a second 
aerial vehicle to our system that will be used initially in a non- 
autonomous mode to test the operations of the ground system 
with the other aerial vehicle. 

Mission Development 

We will use the above capabilities to first develop holarchical 
mission designs and then add emotional models to the 
lMivrdiial"agen'rs“tharwiilTnoveTisioward-bemg-ablrto-supporr 
increasingly complicated mission scenarios. 


Example Mission Scenarios 

To illustrate how these new technologies would be applied, the 
following mission scenario is offered. Two aircraft are sent out 
to image an area, looking for the existence of a specific type of 
mineral or rock. One aircraft is a scout, with long endurance 
and sufficient, but limited sensing capability. The other aircraft 
has a more elaborate sensing capability that weighs more and 
therefore reduces its range or time on station. A ground 
— proces s ings sjisteuB^ - also^ayailable^th a t - jtanJ ak e^ ata^ fafimri he- 
scout or other aircraft and process it in real time to provide 
addition information with which to make decisions. A higher 
level of imaging capability can be applied when both aircraft 
work together and pass their data through the ground system. 

In this example, both aircraft could search separate areas in 
search of the specific mineral. Each aircraft would be 
responsible for maintaining its own safety and resources in 
flight, and also the schedule for the area in which it was 
searching. When either aircraft sensed the mineral, the data 
could be downloaded to the ground system for further 
processing while the aircraft continued to search. A decision to 
reposition the heavily-sensored aircraft to further sense a rock 
found by the scout would depend on multiple resource 
constraints that would have to be resolved individually and 
collectively: (positive identification of the rock sample, position 
of the aircraft and their search schedules, fuel, and other 
constraints). The cooperation and deliberateness of the 
holarchical approach is in contrast with a swarm-type process 
which is comparatively more chaotic and inefficient in nature. 

The mission described above could be developed using a 
standard programming approach. The decision making, 
cooperation, and actions of the individual entities could also be 
implemented using an emotional model. The goals and 
subgoals of each aircraft and its internal modules could be 
modeled as anxieties that would drive it to make certain 
decisions and actions. For example, although the scout aircraft 
could have reported that it found the specified rock, and the 
ground processor may have verified this finding (indicating the 
desire to stereo image that site), the more heavily sensored 
aircraft may have short-term resource or schedule issues that 
had a higher anxiety level at that time. In the case of 
behavioral-based designs, although not straightforward, it may 
be easier to train systems to work together than to program the 
individual intelligent systems and the mission as a whole. 

System development 

We are currently working to add an additional processor and 
sensor capabilities to our target platform to support additional 
onboard reasoning to support holarchical and emotion-based 
approaches. 

In parallel with this effort, a hardware-in-the-loop simulation 
capability is being developed, allowing demonstration and 
testing of algorithms and software in advance of flight-test. 
This software will be enhanced to support real time simulation 
of sensor data to the aircraft while in flight, providing the 
capability to simulate environments and sensor inputs that may 
be quite different from the airport or location in which the 
aircraft is being flown. These virtual capabilities will also aid 
the development and testing of multi-agent behavioral concepts 
before additional platforms are available. 

Upcoming work will enhance the processing and 
communication and capabilities of our ground station. These 
improvements will allow the ground station to exchange data 


We will initially focus on developing a mission holarchy 
between the ground and intelligent aerial vehicle. This should 
shed fight on the operations and interactions needed between 
these differing components. We will then expand the mission 
design to include a second aerial vehicle. We will use this 
design to understand how the holarchy can be enhanced to 
include assets that are similar in operation but are 
heterogeneous in capabilities and objectives. 

In parallel, we will strive to develop an emotional control 
model within a single aerial vehicle with the goal of a simple 
scenario based on recognizing an image and acting in response. 
This basic scenario will involve developing the core emotion 
software, and then deriving and implementing the low and high 
level emotions and control vectors. One of the outputs of the 
development will be an evaluation and comparison of emotional 
vs. conventional control models for achieving similar tasks. 

With a firm understanding of emotional control models, we will 
then work to add an aerial vehicle with this control system to an 
existing mission holarchy. This will involve translating the 
communication and interaction models developed for the 
hierarchy into emotional control equivalents. With success, we 
will expand this development to include other agents in the 
holarchy, taking in consideration the differences between their 
motivations and function in the mission. 

A long-term goal of this effort is to be able to easily define and 
achieve complex missions through assembly, first by defining a 
mission holarchy that includes goals and resources. Then by 
adding emotional agents as those resources that can decompose 
the requirements and act within the hierarchy based on prior 
experience with similar work. 

6. SUMMARY 

In summary, the notion of a holarchy and how this concept 
could be applied to the design of multi-level, heterogeneous 
intelligent systems that work together to solve challenging 
mission objectives has been described. The definition and 
application of the emotional holon concept has been described, 
as well as how it could be constructed within a holarchy and 
used to control the decision-making and actions of a robotic 
system. 

It was also discussed how holarchical concepts and emotional 
decision-making and control could be applied in an example 
mission scenario. Finally, our near term development of 
hardware and software platforms required to support this 
inquiry was described along with a plan for the realization of 
increasing capabilities of holarchy-based missions using 
emotional agents. 
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